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A LATENT-VARIABLE CAUSAL MODEL 
OF FACULTY REPUTATIONAL RATINGS 

ABSTRACT 

Saimier (1985), in an attempt to explain sources of variation in the 
NRC reputational ratings of university faculty, conducted a stepwise 
regrr.ssion analysis using twelve predictor variables. Due to problems 
with multicollinearity and because of the atheoretical nature of stepwise 
regression, Saimier' s conclusions were only speculative. By using LISREL 
the present reanelysis demonstrates the value of regressing reputational 
ratings on three latent variables: size, faculty research productivity, 
and the quality of program graduates. The model was tested using NRC data 
for each of six disciplines: English, French, philosophy, geography, 
political science, and sociology. The relative magnitude of the 
contributions made by the three latent variables depended upon the 
discipline of interest, but generally size and research productivity were 
found to be more important than the quality of the program graduates. 



A LA PENT- VARIABLE CAUSAL MODEL 
OF FACULTY REPUTATIONAL RATINGS 



Quality in graduate programs, as in anything else, is an abstract 
notion that defies precise, objective definition. Professionals in every 
field have a sense of which graduate programs are better than others, but 
it remains unclear exactly what aspects of a department are being 
evaluated by reputational ratings. Several such ratings 
have been published over the years (e.g. , Cartter, 1966; Hughes, 1925, 
1934; Roose & Anderson, 1970). The latest set'Vf reputational ratings, 
sponsored by the National Research Council (Jones, Lindzey, & Coggeshall, 
1982a, 1982b), provides a new set of reputational ratings for many of this 
country* s graduate programs that give research doctorates in fields 
ranging fj^om engineering to the humanities. 

Several researchers (Beyer & Stevens, 1977; Blackburn & 
Lingenfelter, 1973; Elton & Rose, 1972; Hagstrom, 1971; Saunier, 1985) 
have attempted to measure what objective factors contribute to a graduate 
department's reputation. Variables considered to be potential objective 
measures of excellence have included a wide variety of school, department, 
faculty, and student descriptors. As one might expect, these various 
studies reached different conclusions about which objective indicators 
contributed to either the Cartter (1966) or Roose and Anderson (1970) 
ratings. For sociology and political science departments, Beyer and 
Stevens (1977) found that perceived resources was important for 
sociology, but that organization and research productivity were more 



important for political science. Using psychology departments only, 
Elton and Rose (1972) concluded that size was the best predictor. 
Hagstrom (1971), however, found that size, number of citations, and 
undergraduate selectivity were the more important contributors to the 
reputations of mathemetics, physics, chemistry and biology departments. 

In an attempt to better understand the variables that determine the 
reputation of a program's quality as presented in the latest NRC ratings 
(Jones, et al. , 1982a, 1982b), Saunier (1985) regressed the NRC fating 
of quality of graduate faculty on a dozen program, student, and school 
variables included in the NRC reports. Her stepwise regressions were 
conducted with data from each of six disciplines: English, French, 
geography, philosophy, political science, and sociology. Saunier did not 
find the same variables entering into the equations in the same order for 
the six disciplines. She concluded, however, that indicators of 
department size seemed to be the most important predictors of rating. 
Moreover, for all three departments for which publication measures were 
available, at least one such variable entered her equations. 

Because of multicollinearity and the resulting instability of her 
regression coefficients, not to mention the atheoretical approach of 
stepwise regression, Saunier 's (1985) conclusions about the relative 
importance of the variables in explaining reputational ratings could be 
little more than speculative. In her conclusions, Saunier also could not 
directly address the relative importance of the latent variables that she 
alluded to indirectly in her discussion. 

A more useful analytic approach to the study of the objective 
measures of reputational ratings would be to use the variables available 
in Jones, et al. (1982a, 1982b) as indicators of latent variables of 
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theoretical interest. Consider the model shown in Figure 1. The 
variables shown in elipses are latent factors. These are shown to be 
causes of the manifest indicators shown in rectangles. The structural 
portion of the model simply represents a multiple regression equation in 
which faculty ratings are regressed on three exogenous, latent factors: 
size of the department, the quality of the graduate program, and research 
productivity. 



Insert Figure 1 About Here 



Tha purpose of this paper is, accordingly, to estimate the model 
shown in Figure 1 and thus obtain estimates of the relative influences 
of department size, graduate quality, and research productivity on 
departmental reputational ratings. These estimates will provide a more 
parsimonious explanation of faculty reputational ratings than has been 
available heretofore, and also easier to interpret and more substantively 
useful. 



METHODS AND DATA SOURCE 
For purposes of comparison, NRC data (Jones et al. , 1982a, 1982b) 
were selected from the same fields that Saunier (1985) used in her 
analyses: 106 English, 58 French, 77 philosophy, 49 geography, 83 
political science, and 92 sociology departments. Figure 1 shows the model 
tested. The measurement model implied in Figure 1 considers department 
size to be the true cause of three manifest variables: number of faculty 



4 

members in the program in 1980 (FACNUM) , number of graduates from the 
program between 1976 and 1980 (GRADNUM), and the number of full- and 
part-time doctoral students in 1980 (STUDNUM). Quality of graduate 
students in the program is considered to be the underlying cause of a 
single manifest variable, the fraction of graduates who, at the time of 
graduation, had definite employment commitments with Ph. D. -granting 
institutions (EMPPIID)^ The third latent variable, research productivity 
among faculty members, is indexed by the number of published articles 
attributed to program faculty divided by the number of faculty in the 
department (PUBSPER), the proportion of faculty with one or more published 
articles between 1978 and 1980 (MOREIPUB), and the fraction of program 
faculty members holding research grants from the Alcohol, Drug Abuse, and 
Henta] Health Administration, the National Institutes of Health, or the 
National Science Foundation at any time during the FY1978-1980 period 
(GRANTS). For English, French and philosophy departments, the research 
productivity construct is not included since data for the three manifest 
indicators were not available. Accordingly, the research productivity 
factor was dropped from the model for these three disciplines. Estimates 
for the model were obtained with LISREL VI (Joreskog &. Sorbom, 1986). 

RESULTS 

The results of our analyses are shown in Tables 1 and 2. The 
coefficients reported in these tables are maximum likelihood estimates 
produced by the LISREL program. Both metric (and their standard errors) 
and standardized coefficients are given. While the primary purpose of 
the present analysi.s was to provide improved estimates of the structural 
parameters, it is instructive to examine briefly the measurement 
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properties of the model. These results, shown in Table 1, indicate that 
for three departments (English, geography, and political science) the 
number of graduates was the most reliable indicator oi: size (with squared 
correlation coefficients of ,86, ,56, and ,90, respectively). For two 
other departments (French and sociology) the number of students and the 
number of graduates were nearly equal as the most reliable indicators of 
size, while for philosophy, the number of students was the most reliable 
indicator of size. For the three social science departments (geography, 
poli.tical science, and sociology) whose models included the research 
productivity variable, for geography and sociology the most reliable 
indicator of the latent research factor was the number of publications 
per faculty member (with coefficients of ,78 and_, 83, respectively), 
whereas for political science publications per faculty and the fraction 
of faculty with more than one publication between 1978 and 1980 were 
nearly equally reliable. 



Insert Table 1 About Here 



Examination of the standardized structural parameter estimates shown 
in Table 2 indicates that the size of the department is a consistently 
important indicator of departmental reputation. Indeed, for English, 
French, political science, and sociology, size is the most important 
indicator of departmental reputation. The second-most important 
indicator of departmental reputation for the three social science 
departments is faculty research productivity (and foj; geography actually 
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exceeds departmental size in importance by a small margin). The quality 
of the graduate program (as indexed by the proportion of graduates who 
hold positions in Ph.D. -granting institutions) seems not to play an 
important role in the ratings of social science departments; indeed, the 
estimated coefficients for geography and sociology are not even 
statistically significant. Program quality plays a more important role 
among the three humanities departments, but it is unclear whether this 
result is due to the actual importance of the quality of the graduate 
program in determining departmental reputation in the humanities, or due 
to the necessary omission of research productivity from the model for 
these departments. 



Insert Table 2 About Hare 



It also seems evident from an examination of the metric coefficients 
shown iii Table 2 that size, quality of the graduate program, and research 
productivity have differing impacts on reputational ratings across the 
six disciplines. For the humanities, size produces a greater increase 
in the rating of French departments than for English and philosophy, 
whereas the quality of the graduate progi.ttm produces a greater increase 
in the rating of English departments than for French or philosophy. 
Among the social science departments, size and research productivity 
produce a greater increase in ratings for geography than for political 
science and sociology, whereas the quality of the graduate program 



ERIC 



9 



7 

produces a greater increase in the rating of political science departments 
than for geography and sociology. 



DISCUSSION 

Based on these results, it seems clear that size is an important 
determinant of reputational ratings of faculty in the six disciplines 
included in this analysis. Either the larger departments attract 
faculty members of higher reputation, or the faculty within larger 
departments are simply better known. In either event, it does seem that 
part of what is being measured when one rates the reputations of faculty 
is mere quantity; the more faculty, students, and graduates of a 
department, the higher the department's reputation. Tiiis is, of course, 
the same conclusion reached by Saunier (1985), but here we see that the 
effect of size is not uniform across all six disciplines, and indeed is 
sometimes not as important as other objective characteristics of the 
departments. 

For the three social science disciplines (for which publication data 
were available), research productivity is also an important determinant 
of reputational ratings. For geography, research productivity is more 
important than size, and for both political science and sociology it is 
a variable whose effect is at least three-fifths the value of the 
coefficient for size. It would seem, therefore, that research and 
publications provide an important means of exposure to peers in other 
institutions, an exposure reflected in the departmental reputations of 
faculty; which is just another way of saying that a department is known 
by the work of ivs faculty. 
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As for the quality of grad-^ates of a program, this variable seems 
less important, particularly for the social sciences. Apparently, when 
people are asked to rate the reputations of departments, they do so more 
in reference to the size of the department and the research and 
publication productivity of faculty in the department than to the 
characteristics of the students or quality of the graduate program in the 
department. 

In conclusion, it seems clear that two of the most important 
objective determinants of reputational ratings of departments are mere 
size and research and publication productivity. While these are the same 
conclusions reached by Saunier (:.985), the present analysis has been able 
to estimate the relative effects of these latent constructs, and 
demonstrate that their relative influences vary by discipline. This is 
not to say, of course, that other factors are not also being taken into 
account in rating the reputations of departments. As indicated by the 
coefficisnts of determination reported at the bottom of Table 2, there 
is still unexplained variance in the reputational ratings of all six 
disciplines. One possibility may simply be the effect of history; that 
is, some departments may be more highly rated because they have been more 
highly rated in the past, an effect not necessarily reflected in the size 
and productivity of the current depi^rtment. 
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FOOTNOTES 



^ Although Saunier (1985) used several additional variables of 
graduate quality (including fraction of graduates who received a national 
fellowship, the median number of years between enrollment and graduation, 
and the fraction of graduates who had employment) we have used only one 
indicator. Although using all four available indicators results in a 
model that is algebraically identified, in the event the model proved to 
be empirically underidentified due to the small intercorrelations among 
these variables (see Kenny, 1979). Consequerily, the loadings were often 
small, and varied greatly across disciplines, "of ' the four variables 
initially included, the fraction of graduates in Ph. D. -granting 
institutions was consistently the more reliable indicator, and has been 
used here as the only indicator of graduate student quality. 
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Table 1, Maximum Likelihood Measurement Model Parameter 
Estimates for a Model of Faculty Repuf;ational Ratings 
(standard errors in parentheses) 





Discipline 






From To 


English French Philos. 


Geogr. 


PoiiSci 


Socio 1. 




^^etric roeff-ir.iPn-hQ 






SIZE FACNUM 


1. no* 1. 00* 1. 00* 


1. 00* 


1. 00* 


1. OO'*' 


OX oh GRADNUM 


2.88 3.65 1.99 
(.361 (.80) (.51) 


2. 09 
(.47) 


5. 08 
(.79) 


2. 72 
(.36) 


ox lit STuDNuM 


3. 66 5. 24 3. 39 
(.32) (1.14) (.82) 


2. 89 
(.79) 


7.24 
(1. 13) 


4. 32 
(.57) 




1. 00* 1. 00* 1. 00* 


1. 00* 


1. 00* 


1. 00* 


RESPROD PUBSPER 




1. 00* 


1. 00* 


1. 00* 


RESPROD MOREIPUB 




0. 17 .„. 
(.03) 


0. 19 
(.02) 


0. 11 
(.01) 


RESPROD GRANTS 




0. 08 
(.02) 


0. 03 
(.01) 


0. 07 
(.01) 




Correlations Amon? Exopenmif? VflY•^n^.^PQ 


bKAJJlJUAI, SIZE 


.21 .24 .21 


. 35 


. 21 


. 28 


RESrROD SIZE 




.20 


. 23 


. 30 


KhorKUD GRADQUAL 




.40 


.40 


.46 




Sauared Multiple 


Correlations 




FACNUM 


.47 .39 .41 


.48 


. 34 


.58 


GRADNUM 


.86 .64 .39 


.56 


. 90 


.64 


STl'DNUM 


.57 .66 .50 


.35 


. 75 


.64 


EMPPHD 


1.00 1.00 1,00 


1. 00 


1. 00 


1.00 


PUBSPER 




.78 


. 79 


. 83 


MOREIPUB 




.6? 


. 80 


.59 


GRANTS 




.23 


.09 


. 37 


*Fixed parameter. 
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Table 2. Structural Coefficients for Model of Faculty ReputaSional 
Ratings. (standard errors in parentheses) 



LATENT 
VARIABLE 


English 


DISCIPLINE 
Franch Philos. Geogr. 


Pol 1 Sr -f 


^/^n -{ ri1 






Metric Coefficients 








• u oy 
(.010) 


.271* . 148* 
(.062) (.040) 


. 161* 
(. 037) 


. 093* 
(.019) 


. 116* 
(.015) 


Grad Quallty 


4.336* 
(.622) 


1. 988* 2. 935* 
(.846) (.497) 


.498 
(. 638) 


1. 955* 
(.528) 


.589 
(.437) 


Research 







. 744* 
(.134) 


. 408* 
(.118) 


. 437* 
(.075) 






Standardized Coefficients 






Size 


.647 


.720 .470 


.593 


.504 


.668 


Grad Quality 


.414 


.223 .506 


. 073 


.289 


. 08?. 


Research 






.598 


.300 


.424 


R-Square 


.701 


.647 .574 


.918 


.628 


. 868 



* Coefficient is at least twice its standard error. 
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Figure 1. LISREL model of faculty rcputational ratings. 
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